Advances in noise-based testing of concurrent software

Testing of concurrent software written in programming languages like Java and C/C++ is a highly challenging task due to the many possible interactions among threads. A simple, cheap, and effective approach that addresses this challenge is testing with noise injection which influences the scheduling so that different interleavings of concurrent actions are witnessed. In this paper, multiple results achieved recently in the area of noise-injection-based testing by the authors are presented in a unified and extended way. In particular, various concurrency coverage metrics are presented first. Then, multiple heuristics for solving the noise placement problem (i.e., where and when to generate noise) as well as the noise seeding problem (i.e., how to generate the noise) are introduced and experimentally evaluated. In addition, several new heuristics are proposed and included into the evaluation too. Recommendations on how to set up noise-based testing for particular scenarios are then given. Finally, a novel use of the genetic algorithm for finding suitable combinations of the many parameters of tests and noise techniques is presented. Copyright c © 2013 John Wiley & Sons, Ltd.